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Art Unit: 2161 

This is in response to amendment filed April 7, 2005. 

DETAILED ACTION 
Response to Arguments 

1. Applicants arguments with respect to claims 1-12, 14-26, 40-47 and 51-54 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §101 
35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

2. Claims 10-12, 14-26 and 42-47 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

The basis of this rejection is set forth in a two-prong test of: 

(1) whether the invention is within the technological arts; and 

(2) whether the invention produces a useful, concrete, and tangible result. 

For a claimed invention to be statutory, the claimed invention must be within the 
technological art. Mere ideas in the abstract (i.e., abstract idea, law of nature, natural 
phenomena) that do not apply, involve, use, or advance the technological art fail to promote the 
"progress of science and the useful arts" (i.e., the physical sciences as opposed to social sciences, 
for example) and therefore are found to be non- statutory subject matter. For a method claim to 
pass muster, the recited process must somehow apply, involve, use, or advance the technological 
arts. 

As to technological arts recited in the preamble, mere recitation in the preamble (i.e., intended or 
field of use) or mere implication of employing a machine or article of manufacture to perform 
some of the recited steps does not confer statutory subject matter to an otherwise abstract idea 
unless there is positive recitation in the claim as a whole to breathe life and meaning into the 
preamble. In Bowman (Ex parte Bowman, 61 USPQ2d 1665, 1671 (BD. Pat. App. & Inter. 
2001) (Unpublished), the board affirmed the rejection under U.S.C. 101 as being directed to non- 
statutory subject matter. Although Bowman discloses transforming physical media into a chart 
and physically plotting a point on said chart, the Board held that the claimed invention is nothing 
more than an abstract idea, which is not tied to any technological art or environment. 

In the present case, claims 10 and 42 recite a method comprising the steps of identifying 

collecting, classifying and extracting media content, which can be implemented by the mind of a 
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person or by the use of a pencil and paper. In another words, since the claimed invention, as a 
whole, is not within the technological arts as explained above, these claims only constitute an 
idea and does not apply, involve, use, or advance the technological arts, thus, it is deems to be 
directed to non- statutory subject matter. 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Marchisio (U. 
S. Patent no. 6,510,406) in view of Diamond (U. S. Patent no. 6,269,368) and further in view of 
Wang etal. (US 5,802,361). 

Regarding claim 1, Marchisio discloses one or more computer-readable media having 
stored thereon a plurality of instructions that, when executed by one or more processors of a 
computer, causes the one or more processors to perform the following acts: 

- receiving search criteria (see col. 5, lines 8-11 and col. 7, lines 27-34, Marchisio); 

- generating a query vector based on text features of the search criteria (see col. 5, lines 
1 1-17 and col.7, lines 35-39 and step 17, Fig.l, Marchisio); 

- identifying media content pieces (see col. 19, lines 10-13 and Fig. 12, Marchisio). 
Please note that the "digitized speech" corresponds to "media content"; 

- modifying the query vector based on the user feedback (see col.7, lines 55-65 and col. 
16, lines 52-56, Marchisio); 
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- modifying one or more of the text feature vectors (see feature 27, Fig.3 and col. 14, 
lines 25-31, Marchisio). 

identifying new media content pieces (see col. 7, lines 30-34; 27, Fig. 3). 
Marchisio, however, does not explicitly disclose "comparing the query vector to text 
feature vectors" and "receiving user feedback regarding the relevancy of the identified 
media content pieces". Diamond, on the other hand, discloses an information retrieval 
using dynamic evidence combination system including comparing the query vector to 
text feature vectors and receiving user feedback (see col. 6, line 65 to col. 7, line 5 and 
lines 9-11; col. 18, line 50 to col. 19, line3, Diamond). It would have been obvious to one 
of ordinary skill in the art at the time of the invention to modify Marchsio to include the 
claimed features as taught by Diamond. The motivation of doing so would have been to 
obtain the desire of information retrieval technique that capture both the preciseness and 
richness of meaning in queries and documents (see col. 2, lines 3-6, Diamond). 
Marchisio and Diamond does not disclose "non-text media" and "classifying the one or 
more of non-text media content". Wang discloses a method for searching graphic images 
and videos including the step of classifying the images that corresponds to "non-text 
media" (col.l, lines 7-10 and col.2, lines 57-61, Wang). It would have been obvious to 
one of ordinary skill in the art at the time of the invention was made to modify the 
combination system of Marchisio and Diamond to include the claimed feature as taught 
by Wang. The motivation of doing so would have been to provide the user a simple and 
rapid way to specify and modify the non-text media (col.4, lines 20-39, Wang). 
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Regarding claim 2, Marchisio/DiamondAVang combination further discloses generating 
another query vector based on one or more low-level features of the search criteria (see col. 17, 
lines 8-21, Marchisio); and wherein the identifying comprises, comparing the query vector to 
text feature vectors associated with the plurality of media content pieces to generate first results, 
comparing the other query vector to other low-level feature vectors associated with the plurality 
of media content pieces to generate second results, and combining, for one of the plurality of 
media content pieces, the first and second results corresponding to the one media content piece 
(see col. 17, lines 47-53 and Fig. 10, Marchisio). 

Regarding claim 3, Marchisio/Diamond/Wang combination further discloses the altering, 
based on the user feedback, a weighting of the results used in the combining (see col. 6, line 67 
to col. 7, line 8 and col. 10, lines 30-39, Diamond). 

Regarding claim 4, Marchisio/DiamondAVang combination further discloses the step of 
determining and weighting the result corresponding to the media content piece (see col. 17, lines 
54-66, Marchisio). 

Regarding claim 5, Marchisio/Diamond/Wang combination further discloses the step of 
altering a weighting of one or more elements in the feature vector based on the user feedback 
(see col. 19, lines 10-19, Diamond). 

Regarding claim 6, Marchisio/Diamond/Wang combination further discloses the search 
criteria comprise one or more words (see 5, Fig.l, Marchisio). 

4. Claims 7-9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Marchisio (U. 

5. Patent no. 6,510,406) in view of Diamond (U. S. Patent no. 6,269,368) and further view of 
Hoffert et al. (U. S. Patent no. 6,282,549). 
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Regarding claim 7, Marchisio/Diamond disclose all of the claimed limitation as discussed 
above except that the piece of media content comprises an image. Hoffert, on the other 
hand, discloses the web crawler to search and locate media files including images (see 
101, Fig.l and col. 3, lines 28-54, Hoffert). It would have been obvious to one of ordinary 
skill in the art at the time of the invention to modify Marchsio to include the crawler 
module couple to access media content as evidenced by Hoffert. The motivation of doing 
so would have been to allow user to search media content which Marchisio would not be 
able to (see col. 2, lines 24-27, Hoffert). 

Regarding claim 8, Marchisio/Diamond/Hoffert combination further discloses the piece 
of media content comprises a portion of audio content (see col. 8, lines 1-53 and 101, Fig.l, 
Hoffert). 

Regarding claim 9, Marchisio/Diamond/Hoffert combination further discloses the piece 
of media content comprises a portion of multimedia content (see col. 8, lines 1-53 and abstract of 
Hoffert). 

5. Claims 10-12 and 14-26 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Hoffert et al. (U. S. Patent no. 6,282,549) in view of Wang et al. (US 5,802,361). 
Regarding claims 10 and 26, Hoffert discloses a method comprising: 

- identifying a media content source (see col. 3, lines 14-21, Hoffert). 

- Hoffert discloses the web crawler to search and index for media files and find media 
content located at the web site of interest and associated with the references of the 
media files (see col. 3, lines 28-54 and col. 4, lines 18-20 and 45-47, Hoffert). This 
reference corresponds to the "associated text" from the content source. These 
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searching and indexing reads on "collecting one or more pieces of media content and 
associated text from the media content source"; 

- extracting, for a piece of media content, one or more text features from the associated 
text (see col.18, lines 40-45 and col.21, lines 17-22, Hoffert). The "useful 
information" or "meaningful information" corresponds to the "text features" that is 
extracted from the data, according to col. 7, lines 48-67 of Hoffert, this data is "the 
associated text"; and 

- making the one or more text features available for searching (see col.18, lines 40-45 
and col.21, lines 17-22, Hoffert). 

Hoffert does not disclose "non-text media" and "classifying the one or more of non-text 
media content". Wang discloses a method for searching graphic images and videos 
including the step of classifying the images that corresponds to "non-text media" (col.l, 
lines 7-10 and col.2, lines 57-61, Wang). It would have been obvious to one of ordinary 
skill in the art at the time of the invention was made to modify Hoffert to include the 
claimed feature as taught by Wang. The motivation of doing so would have been to 
provide the user a simple and rapid way to specify and modify the non-text media (col.4, 
lines 20-39, Wang). 

Regarding claim 11, Hoffert/Wang combination further disclose generating one or more 
text feature vectors from the extracted one or more text features (see col. 21, lines 20-22, 
Hoffert); and wherein the making comprises making the one or more text feature vectors 
available for searching (see col.18, lines 40-45 and col. 21, lines 17-22, Hoffert). 
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Regarding claim 12, Hoffert/Wang combination further teaches the level of content 
attributes to be meaningful which corresponds to "low-level feature" (col. 21, lines 14-15); and 
making the one or more low-level features available for searching (see col. 18, lines 40-45, 
Hoffert). 

Regarding claim 14, Hoffert/Wang combination further disclose a plurality of web pages, 
each web page including a plurality of pieces of media content and text associated with one or 
more of the plurality of pieces of media content (col. 4, line 8 and lines 55-57, Hoffert). 

Regarding claim 15, Hoffert/Wang combination further disclose a filename and the one 
or more text features comprises one or more words in the filename (see col. 6, line 22, Hoffert). 

Regarding claim 16, Hoffert/Wang combination further disclose a uniform resource 
locator (URL) and the one or more text features comprise one or more words in the URL (see 
col. 6, lines 45-67, Hoffert). 

Regarding claim 17, Hoffert/Wang combination further disclose that the associated text 
for a piece of media content comprises alternate text that can be displayed in place of the media 
content, and the one or more text features comprises one or more words of the alternate text (see 
col. 5, lines 30-34, Hoffert). 

Regarding claim 18, Hoffert/Wang combination further disclose the associated text for a 
piece of media content comprises text surrounding the piece of media content on a web page, and 
the one or more text features comprises one or more words of the text surrounding the piece of 
media content (see col. 5, lines 62 to col. 6, lines 9, Hoffert). 

Regarding claim 19, Hoffert/Wang combination further disclose the associated text for a 
piece of media content comprises a title of a web page that includes the piece of media content, 
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and the one or more text features comprises one or more words in the title (see col. 6, lines 19- 
22, Hoffert). 

Regarding claim 20, Hoffert/Wang combination further disclose the associated text for a 
piece of media content comprises a link on a web page that includes the piece of media content, 
and the one or more text features comprises one or more words in the link (see col. 6, lines 15- 
21, Hoffert). 

Regarding claim 21, Hoffert/Wang combination further disclose the associated text for a 
piece of media content comprises anchor text corresponding to the piece of media content, and 
the one or more text features comprises one or more words in the anchor text (see col. 7, lines 
46-61, Hoffert). 

Regarding claim 22, Hoffert/Wang combination further disclose the associated text for a 
piece of media content comprises an image annotation corresponding to the piece of media 
content, and the one or more text features comprises one or more words in the image annotation 
(see col. 7, lines 45-51, Hoffert). 

Regarding claim 23, Hoffert/Wang combination further disclose that the piece of media 
content comprises an image (see 101, Fig.l, Hoffert). 

Regarding claim 24, Hoffert/Wang combination further disclose the piece of media 
content comprises a portion of audio content (see col. 8, lines 1-53 and 101, Fig.l, Hoffert). 

Regarding claim 25, Hoffert/Wang combination further disclose the piece of media 
content comprises a portion of multimedia content (see col. 8, lines 1-53 and abstract of Hoffert). 
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6. Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over Marchisio (U. S. 
Patent no. 6,510,406) in view of Hoffert et al. (U. S. Patent no. 6,282,549) further in view of 
Wang et al (US 5,802,361). 

Regarding claim 40, Marchisio discloses a system comprising: 

a feature extraction module (21, Fig2) coupled to extract for a piece of the one 
or more different types and from the one or more text features from one of the 
media content pieces (see col. 8, lines 46-49, Marchisio); and 
a content indexing module (20, Fig.2) coupled to generate, for a piece, a text 
feature vector (see col. 5, lines 8-14, col. 6, lines 35-38, Marchisio), based on 
the extracted one or more text features, corresponding to the one media content 
piece (see col. 8, lines 50-56, Marchisio). 
Marchisio, however, does not explicitly disclose "a crawler module coupled to access a 
media content source and collect a plurality of media content pieces and associated text 
from the media content source." Hoffert, on the other hand, discloses the web crawler to 
search, locate media files and find media content located at the web site of interest and 
associated with the references of the media files (see col. 3, lines 28-54 and col 4, lines 
18-20 and 45-47, Hoffert). This reference corresponds to the "associated text" from the 
content source. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to modify Marchsio to include the crawler module couple to 
access media content as evidenced by Hoffert. The motivation of doing so would have 
been to allow user to search media content which Marchisio would not be able to (see 
col. 2, lines 24-27, Hoffert). 
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Marchisio and Hoffert does not disclose "non-text media" and "classifying the one or 
more of non-text media content". Wang discloses a method for searching graphic images 
and videos including the step of classifying the images that corresponds to "non-text 
media" (col.l, lines 7-10 and coL2, lines 57-61, Wang). It would have been obvious to 
one of ordinary skill in the art at the time of the invention was made to modify the 
combination system of Marchisio and Hoffert to include the claimed feature as taught by 
Wang. The motivation of doing so would have been to provide the user a simple and 
rapid way to specify and modify the non-text media (col.4, lines 20-39, Wang). 

7. Claim 41 is rejected under 35 U.S.C. 103(a) as being unpatentable over Diamond (U. S. 

Patent no. 6,269,368) in view of Hoffert et al. (U. S. Patent no. 6,282,549) and further in view of 

Wang et al. (US 5,802,361). 

Regarding claim 41, Diamond discloses a system somprising: 

- a query generator (50, Fig. 1 , Diamond) to generate a query vector based on received 
search criteria (see col. 6, lines 25-33, Diamond). Please note that "representation of 
query" corresponds to the "query vector"; and 

a matching module (55, Fig. 1 , Diamond), to receive the query vector and compare 
the query vector to a plurality of feature vectors (see col. 6, lines 37-42 and col. 18, 
lines 62-66, Diamond). Please note that "representation of document" corresponds to 
the "feature vector". 

Diamond, however, does not disclose "a plurality of pieces of media content". Hoffert, on 
the other hand, discloses the web crawler to search and locate multimedia files (see 
abstract, element 101, Fig.l and col. 3, lines 28-54, Hoffert). It would have been obvious 
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to one of ordinary skill in the art at the time of the invention to modify Diamond to 
include the plurality of media content pieces as evidenced by Hoffert. The motivation of 
doing so would have been to allow user to search media content which Diamond would 
not be able to (see col. 2, lines 24-27, Hoffert). 

Diamond and Hoffert does not disclose "non-text media" and "classifying the one or 
more of non-text media content". Wang discloses a method for searching graphic images 
and videos including the step of classifying the images that corresponds to "non-text 
media" (col.l, lines 7-10 and col.2, lines 57-61, Wang). It would have been obvious to 
one of ordinary skill in the art at the time of the invention was made to modify the 
combination system of Diamond and Hoffert to include the claimed feature as taught by 
Wang. The motivation of doing so would have been to provide the user a simple and 
rapid way to specify and modify the non-text media (col.4, lines 20-39, Wang). 

8. Claims 5 1-54 are rejected under 35 U.S.C. 103(a) as being unpatentable over Diamond 

(U. S. Patent no. 6,269,368) in view of Hoffert etal. (U. S. Patent no. 6,282,549) and further in 

view of Schuetze et al. (U. S. Patent no. 6,567,797). 

Regarding claim 51, Diamond discloses one or more computer-readable media having 

stored thereon a plurality of instructions that, when executed by one or more processors of a 

computer, causes the one or more processors to perform acts including: 

- comparing a query vector corresponding to search criteria of the user and a feature 
vector (see col. 6, lines 37-42 and col. 18, lines 62-66, Diamond). Please note that 
"representation of query and document" corresponds to the "query and feature 
vector"; 
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- receiving user feedback regarding the relevancy of data content (see col. 18, lines 34- 
55, Diamond); 

- modifying the query vector based on the received user feedback (see col. 19, lines 39- 
46, Diamond). 

Hoffert, on the other hand, discloses the method of searching and indexing media content 
including media content piecess (see 101, Fig.l and col. 3, lines 28-54, Hoffert). It would 
have been obvious to one of ordinary skill in the art at the time of the invention to modify 
Diamond to include media content as evidenced by Hoffert. The motivation of doing so 
would have been to allow user to search media content which Diamond would not be able 
to (see col. 2, lines 24-27, Hoffert). 

Diamond and Hoffert combination does not disclose "the received user feedback in an off 
line". Schuetze, on the other hand, discloses a system for retrieving information from a 
collections of uses multi-modal features of the documents in the collection including the 
user feedback processing that is performed off-line (abstract; summary and col. 8, lines 5- 
14, Schuetze). It would have been obvious to one of ordinary skill in the art at the time of 
the invention to include the user feedback in an off-line as taught by Schuetze. The 
motivation of doing so would have been to provide flexibility and useful searching based 
on the text feature vector (col. 5, lines 30-39, Schuetze). 

Regarding claims 52 and 53, Diamond/Hoffert/ Schuetze combination discloses all of the 
claimed subject matter. However, Diamond's "values to the hidden layer" (col. 17, line 29, 
Diamond) reads on the claimed "confidence" value. Because the new query vector D is adjusted 
based on the relevancy value (in the same manner as is the claimed vector D adjusted on the 
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confidence value), the Examiner takes the position that the claimed relationship (see formula 
claim 52) is in fact inherent in Marchisio. And for the formula in the claim 53 is basically a 
calculating the different between the value 1 and adjusted vector. 

Regarding claim 54, Diamond/Hoffert/ Schuetze combination discloses that the pieces of 
media content comprise the audio content, visual content, and multimedia content (see Fig. 1 and 
Abstract ofHoffert). 

Allowable Subject Matter 

9. Claims 42-47 would be allowable if rewritten to overcome 101 rejection set forth above. 

10. The following is a statement of reasons for the indication of allowable subject matter: 
The prior arts fail to disclose or suggest "comparing a high-level query vector to a high-level 
feature vector of the other piece of media content and a result of comparing a low-level query 
vector to a low-level feature vector of the other piece of media content and combining the weight 
result to determine whether to identify the other piece of media content for rendering." 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Foote et al. (US 6,404,925 Bl) disclose methods and apparatuses for segmenting an 
audio-visual recording using image similarity searching and audio speaker recognition. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Hanh B. Thai whose telephone number is 571-272-4029. The 
examiner can normally be reached on 8 AM - 4:30 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on 571-272-4023. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Hanh B Thai 
Examiner 
Art Unit 2161 



June 27, 2005 
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